20 research outputs found
The Logic of Random Pulses: Stochastic Computing.
Recent developments in the field of electronics have produced nano-scale devices whose operation can only be described in probabilistic terms. In contrast with the conventional deterministic computing that has dominated the digital world for decades, we investigate a fundamentally different technique that is probabilistic by nature, namely, stochastic computing (SC). In SC, numbers are represented by bit-streams of 0's and 1's, in which the probability of seeing a 1 denotes the value of the number. The main benefit of SC is that complicated arithmetic computation can be performed by simple logic circuits. For example, a single (logic) AND gate performs multiplication. The dissertation begins with a comprehensive survey of SC and its applications. We highlight its main challenges, which include long computation time and low accuracy, as well as the lack of general design methods. We then address some of the more important challenges. We introduce a new SC design method, called STRAUSS, that generates efficient SC circuits for arbitrary target functions. We then address the problems arising from correlation among stochastic numbers (SNs). In particular, we show that, contrary to general belief, correlation can sometimes serve as a resource in SC design. We also show that unlike conventional circuits, SC circuits can tolerate high error rates and are hence useful in some new applications that involve nondeterministic behavior in the underlying circuitry. Finally, we show how SC's properties can be exploited in the design of an efficient vision chip that is suitable for retinal implants. In particular, we show that SC circuits can directly operate on signals with neural encoding, which eliminates the need for data conversion.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113561/1/alaghi_1.pd
Neural Network Compression for Noisy Storage Devices
Compression and efficient storage of neural network (NN) parameters is
critical for applications that run on resource-constrained devices. Although NN
model compression has made significant progress, there has been considerably
less investigation in the actual physical storage of NN parameters.
Conventionally, model compression and physical storage are decoupled, as
digital storage media with error correcting codes (ECCs) provide robust
error-free storage. This decoupled approach is inefficient, as it forces the
storage to treat each bit of the compressed model equally, and to dedicate the
same amount of resources to each bit. We propose a radically different approach
that: (i) employs analog memories to maximize the capacity of each memory cell,
and (ii) jointly optimizes model compression and physical storage to maximize
memory utility. We investigate the challenges of analog storage by studying
model storage on phase change memory (PCM) arrays and develop a variety of
robust coding strategies for NN model storage. We demonstrate the efficacy of
our approach on MNIST, CIFAR-10 and ImageNet datasets for both existing and
novel compression methods. Compared to conventional error-free digital storage,
our method has the potential to reduce the memory size by one order of
magnitude, without significantly compromising the stored model's accuracy.Comment: 19 pages, 9 figure
Survey of Stochastic Computing
Stochastic computing (SC) was proposed in the 1960s as a low-cost alternative to conventional binary computing. It is unique in that it represents and processes information in the form of digitized probabilities. SC employs very low-complexity arithmetic units which was a primary design concern in the past. Despite this advantage and also its inherent error tolerance, SC was seen as impractical because of very long computation times and relatively low accuracy. However, current technology trends tend to increase uncertainty in circuit behavior, and imply a need to better understand, and perhaps exploit, probability in computation. This paper surveys SC from a modern perspective where the small size, error resilience, and probabilistic features of SC may compete successfully with conventional methodologies in certain applications. First, we survey the literature and review the key concepts of stochastic number representation and circuit structure. We then describe the design of SC-based circuits and evaluate their advantages and disadvantages. Finally, we give examples of the potential applications of SC, and discuss some practical problems that are yet to be solved